A Feature Selection Method based on Fuzzy Mutual Information for Fuzzy Rule-based Regression Models

Fuzzy rule-based models have been extensively used in regression problems. Besides high accuracy, one of the most appreciated characteristics of these models is their interpretability, which is generally measured in terms of complexity. Complexity is affected by the number of features used for generating the model: the lower the number of features, the lower the complexity. Feature selection can therefore considerably contribute not only to speed up the learning process, but also to improve the interpretability of the final model. Nevertheless, a very few methods for selecting features before rule learning have been proposed in the literature in the framework of regression problems. In this context, we propose a novel forward sequential feature selection approach based on the minimalredundancy-maximal-relevance criterion. The relevance and the redundancy of a feature are measured in terms of, respectively, the fuzzy mutual information between the feature and the output variable, and the average fuzzy mutual information between the feature and the just selected features. The stopping criterion for the sequential selection is based on the average values of relevance and redundancy of the just selected features. We tested our feature selection method performing two experiments on twenty regression datasets. In the first experiment, we aimed to show the effectiveness of our approach by comparing the mean square errors achieved by the fuzzy rule-based models generated using all the features, the features selected by our approach and the features selected ∗Corresponding author, Tel: +39 0502217678 Fax: +39 0502217600 Preprint submitted to Information Science December 23, 2014 by two state-of-the-art feature selection algorithms, respectively. For simplicity, we adopted the well-known Wang and Mendel algorithm for generating the fuzzy rule-based models. We present that the mean square errors obtained by models generated by using the features selected by our approach are on average similar to the values achieved by using all the features and lower than the ones obtained by employing the subset of features selected by the two state-of-the-art feature selection algorithms. In the second experiment, we intended to evaluate how our feature selection algorithm can reduce the convergence time of the evolutionary fuzzy systems, which are probably the most effective fuzzy techniques for tackling regression problems. By using a state-of-the-art multi-objective evolutionary fuzzy system based on rule learning and membership function tuning, we show that the number of evaluations can be reduced of more than 40% when pre-processing the dataset by our feature selection algorithm.